The present invention relates to a stream data processing system operable in such a way that stream data (moving picture data, musical sound data and the like) is entered (captured) thereinto from an input device, a certain process operation is carried out with respect to the captured stream data, and thereafter, the processed stream data is outputted from a separate output device from the input device, and further relates to a stream data processing method, stream data processing program, and computer readable recording medium for storing the stream data processing program.
While stream data such as multimedia data are processed, data stream is synchronized based upon certain timing information and is input and output, and processed. As the timing information, physical clocks, and time stamp information contained in certain stream data are provided.
On the other hand, in such stream data processing apparatus, there are possibilities that both input devices into which the stream data are entered, and output devices which output the inputted stream data to external devices are operated in response to different clocks from each other. For example, even if both the input device and the output device are to be operated in response to same clock of 44.1 kHz, there is a possibility that operating clock periods of the input device and the output device are slightly deviated each other due to using different oscillators. Accordingly, the clocks used for the input devices are not sometimes synchronized with the clocks used for the output devices. As a consequence, during data transmission between the devices, since data are overflown, or are conversely depleted, such a problem may occur. That is, output data may not own desirable formats.
These problems may also occur in a case that the stream data are constituted by software. Further, recently, in order to process stream data by personal computers (PCs) in software manners, in the operating system, for example, Windows 98 (registered trademark) provided by Microsoft corporation, the API (Application Program Interface) suitable for processing the stream data on the PCs has been proposed. That is, this API is referred to as “DirectShow (registered trademark)” system, and is utilized so as to connect the PCs to external appliances by employing USB (universal Seria Bus) and/or IEEE 1394 interfaces. In this “DirectShow” system, modules (called as “filters”) capable of executing certain process operations on data are mutually connected to each other in order to execute a desirable data processing operation.
In such a data processing system constituted by these mutually-connected software filters, in the case that the operating clocks of each filter are slightly deviated each other, output data may not have a desirable format due to overflow or depletion of the data.
In the case that a portion of these filters is formed in a user mode or in the case that the portion of the filters is formed in a kernel mode but priority thereof is low, output data may not have a desirable format due to overflow or depletion of the data. For example, in many of the cases, an effector filter contained in an audio data processing system is formed in a user mode since the system has an user interface function which allows the user to set effects. It should be noted that this “kernel mode” implies such an operation mode to which a very high priority is given in an operating system (OS) such as WINDOWS (registered trademark) of Microsoft Corporation, namely such an operation mode that a code can directly access all of hardware and also all of memories. On the other hand, this “user mode” implies such an operation mode whose priority is set to a low level in the OS such as WINDOWS (registered trademark), namely such an operation mode that a code cannot directly access hardware.
In this case, the below-mentioned problem may occur. That is, since the audio data transmission between a filter formed in a kernel mode and a filter formed in a user mode and/or between filters formed in the user mode takes large load for processing in comparison with the data transmission between the filters formed in the kernel mode, throughput of the effector filter varies due to influence from other task, and the audio data may be overflown, or depleted before/after the effector filter, so that stream data cannot be outputted in an ideal format, and/or noise may be produced.
Situation in which the throughput of the effector filter varies due to the influence from other task may be occurred in the case that data is created in low priority even if the filter is in the kernel mode.
To solve the above-described problem, for instance, Japanese Patent Publication No. Hei-10-283199 discloses the synchronizing apparatus. This conventional synchronizing apparatus is designed to minimize lag of the output timing of the plural stream data in such a manner that while this synchronizing apparatus owns three sets of different time values, i.e., the positional time value, the physical time value, and the relative time value, these three time values are commonly utilized in large numbers of devices. This positional time value corresponds to such a time value produced based upon time interval information related to a data stream, and reflects a position of the data stream to be processed. The physical time value corresponds to such a time value produced based upon a hardware oscillator, or a clock. The relative time value is to provide a designated time value such as the positional time value in connection with a reference time value.
However, this conventional synchronizing apparatus disclosed in the above-described Japanese Patent Publication can minimize deviation of input and output timing. However, since three different time values are used, complex control is needed for adjustment of processing rate, etc.